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A few  decades  ago,  it  was  common  to  use  exams  as  the  single  measure  of  performance  in  an 
introductory  taxation  course.  Median  scores  in  the  mid-60s  on  these  exams  were  not  uncommon, 
results  that  were  depressing  and  frustrating  for  students  and  professors.  Although  students  were 
assigned  reading  and  homework  assignments,  not  all  of  them  kept  current  with  this  work  if  it  was  not 
graded.  Cramming  for  exams  was  commonplace.  Questions  asked  in  fits  of  desperation  moments 
before  exams  generally  foretold  poor  performance,  which  contributed  to  a sense  of  hopelessness.  A 
pedagogical  and  cultural  change  was  badly  needed. 

Through  the  years,  tax  return  preparation  projects  were  added,  and  in-class  group  projects  were 
attempted  long  before  the  phrase  "flipping  the  classroom"  became  widespread  (see  Franklin,  "Flipping 
the  Tax  Classroom  for  a Better  Student  Learning  Experience  (/issues/201 5/nov/flippinq-tax- 
classroom.html)."  46  The  Tax  Adviser  852  (November  2015)).  In  addition,  as  a means  of  improving 
preparation  for  class  meetings,  instructors  started  collecting  homework  assignments  from  a random 
selection  of  students  each  class  period.  Although  the  latter  helped  improve  students'  preparation  for 
class  to  a degree,  some  students  took  advantage  of  the  lottery  system,  gambling  that  they  would  not 
be  selected.  Moreover,  this  system  did  not  allow  for  immediate  feedback  and  could  not  be  used  to 
plan  the  current  class  period's  discussion.  Yet,  homework  graded  merely  for  satisfactory  effort, 
selected  at  random  intervals — a very  noisy  measure  of  effort — often  correlated  highly  with  exam 
results.  Exam  results  were  better  than  before  these  changes  were  implemented,  but  most  of  the 
improvement  was  achieved  by  high  or  above-average  achievers.  These  students  likely  already  had 
good  habits  and  were  merely  being  rewarded  for  them.  Average  students  failed  to  share  in  the 
improvement.  Too  many  students  viewed  homework  as  a busy-work  task. 

Measurable  improvement  finally  came  when  homework  was  replaced  with  online  quizzes — both  pre- 
quizzes (before  classroom  coverage  of  a topic)  and  post-quizzes  (after  classroom  coverage).  The 
online  quizzes  make  every  student  accountable  for  each  class  meeting,  encourage  distributed 
practice  (as  opposed  to  cramming)  that  allows  for  reflection  and  correcting  errors,  inform  instructional 
decisions,  and  allow  the  professor  opportunities  to  model  decision  strategies  in  the  classroom. 

Exams  used  in  the  course  today  are  far  more  difficult  and  complex  than  exams  used  two  decades 
ago,  but  performance  is  vastly  superior,  with  medians  around  80%  or  better  becoming  common. 


Implementation  of  Pre-Quizzes 


One  of  the  authors  has  implemented  online  pre-quizzes  over  10  semesters  in  an  introductory  tax 
course  at  Kansas  State  that  meets  two  times  per  week.  Before  each  class  meeting,  students  are 
required  to  complete  a five-question  multiple-choice  quiz  on  the  material  assigned  for  that  class 
meeting.  At  the  beginning  of  the  semester,  students  are  given  a detailed  assignment  schedule  that 
provides  a reading  list  of  topics  that  will  be  covered  for  each  class  meeting,  plus  information 
contained  in  the  textbook  (including  discussion  questions  and  problems),  the  instructor-prepared 
class  notes,  and/or  the  applicable  sections  of  the  Internal  Revenue  Code  and  Treasury  regulations. 

Each  quiz  is  time-sensitive.  The  pre-quiz  becomes  available  once  the  third  of  three  sections 
commences  (e.g.,  4 p.m.  on  Tuesday)  and  must  be  completed  before  the  first  section  reconvenes 
(e.g.,  11:30  a.m.  on  Thursday).  Each  quiz  consists  of  five  questions  based  on  the  assigned  material. 
The  questions  mix  conceptual,  numerical,  and  research -related  issues  (requiring  students  to  review 
the  applicable  material  in  the  Code  and  regulations).  All  quiz  questions  are  written  by  the  instructor 
and  customized  to  the  covered  material. 

The  questions  are  randomized  to  the  students  and  must  be  completed  within  five  minutes.  Each 
question  is  worth  0.5  points,  so  each  quiz  is  worth  2.5  points.  Twenty-five  pre-quizzes  are  assigned 
during  the  semester.  The  best  scores  from  20  quizzes  (50  potential  points)  are  included  in  the 
student's  semester  total  and  constitute  6.5%  of  each  student's  final  grade.  The  solution  for  each  quiz 
and  performance  statistics  are  made  available  after  class  to  provide  students  with  feedback  on  their 
quiz  performance. 

The  purpose  of  the  pre-quiz  for  students  is  twofold.  First,  the  pre-quizzes  motivate  students  to 
engage  in  the  course  material  before  the  class  meetings.  Toward  this  end,  the  quizzes  are  designed 
to  achieve  a median  score  of  2.0.  Although  the  questions  are  far  from  simple,  students  feel  rewarded 
for  being  accountable.  Second,  the  pre-quizzes  allow  students  to  reflect  on  what  they  have  studied 
and  make  them  receptive  to  what  will  occur  in  the  class  meeting.  The  feedback  they  receive  allows 
students  to  modify  their  processes  as  they  seek  to  improve. 

For  the  professor,  the  benefits  also  are  twofold.  First,  all  results  are  reviewed  before  the  classroom 
meeting,  and  areas  of  weakness  or  common  misconceptions  are  used  as  guides  for  the  ensuing 
classroom  discussion.  Second,  the  pre-quiz  questions  are  used  to  model  decision  strategies.  The 
questions  are  regularly  presented  in  class  and  analyzed,  and  decision  processes  are  discussed.  A 
great  deal  of  student  difficulty  in  the  tax  area  stems  from  an  inability  to  structure  the  analysis. 
Emphasis  is  placed  on  schema  development  so  that  students  have  more  confidence  applying  the 
material  to  different  situations.  The  focus  of  the  feedback  then  is  on  learning  and  continual 
improvement,  rather  than  on  a grade. 

Post-Quiz  Follow-Up 

Students  also  complete  a post-quiz  each  week  to  examine  what  has  been  covered  in  class.  The 
post-quiz  is  made  available  each  Tuesday  night  and  must  be  completed  before  the  first  class 
reconvenes  on  Thursday.  Each  quiz  consists  of  nine  objective  questions  worth  one  point  each. 
However,  the  quizzes  are  treated  as  being  worth  eight  points,  allowing  the  students  to  potentially  earn 
an  extra  point.  The  post-quizzes  constitute  a little  over  10%  of  each  student's  semester  grade. 


Having  the  post-quizzes  scored  creates  an  incentive  for  students  to  perform  at  a high  level,  but 
because  the  quizzes  allow  a margin  for  error,  students  come  to  see  the  post-quizzes  as  practice — a 
chance  to  make  mistakes  and  learn  from  them.  By  being  repeatedly  quizzed  on  the  material,  students 
are  forced  to  be  intellectually  honest  with  themselves.  This  breaks  the  destructive  habit  of  believing 
that  something  is  understood  when,  in  fact,  knowledge  is  incomplete  or  suspect. 

As  with  the  pre-quizzes,  performance  is  reviewed  by  the  professor  before  the  next  class,  allowing  for 
use  of  class  time  to  correct  perceived  issues.  Many  times,  the  most  challenging  quiz  questions  are 
presented  in  class,  allowing  decision  strategies  to  be  modeled  by  the  professor  and  queries  to  be 
received  from  the  students.  In  all  cases,  a detailed  explanation  for  each  question  is  posted  online 
after  class  so  that  students  can  receive  immediate  feedback. 

Benefits  and  Uses 

The  benefits  of  repeated  online  quizzes  are  many.  First,  students  come  to  class  better  prepared  and 
engaged.  It  is  common  to  hear  students  discussing  some  of  the  quiz  questions  among  themselves 
before  class  begins.  Students  are  more  inquisitive  during  class  and  frequently  relate  what  is  being 
discussed  or  presented  back  to  what  they  had  been  asked.  In  other  words,  they  become  active, 
rather  than  passive,  learners. 

Second,  by  encouraging  distributed  practice  and  reducing  mass  practice  (i.e.,  cramming),  students 
demonstrate  increased  mastery  of  the  subject  matter.  The  benefit  observed  is  greatest  among 
average  students,  many  of  whom  discover  through  this  process  how  to  better  direct  their  own 
learning.  Spreading  the  practice  allows  students  time  to  reflect  on  what  they  are  learning,  correct 
mistakes  and  misperceptions,  and  begin  to  internalize  what  they  are  encountering.  Simply  put,  each 
time  they  work  on  an  issue,  a little  more  becomes  clear. 

Students  need  to  have  opportunities  to  make  mistakes,  seek  corrections,  and  even  at  times  to  try 
new  and  different  ways  of  preparing  the  material.  The  use  of  pre-  and  post-quizzes  allows  students  to 
"stair-step"  toward  the  exams.  Many  of  the  authors'  students  have  a year  gap  between  the 
introductory  tax  course  and  the  tax  courses  in  the  Master  of  Accountancy  program.  Despite  this  time 
lag,  the  authors  observe  excellent  recall  and  retrieval  by  students,  providing  evidence  that  the 
material  in  the  introductory  course  was  ingrained  into  long-term  memory. 

Third,  student  feedback  is  positive.  Although  students  initially  complain  about  the  workload,  they 
embrace  the  process  once  they  see  the  benefits.  Students  have  a much  greater  feel  for  their 
progress,  and  they  know  where  to  spend  time  to  improve. 

Having  been  forced  away  from  cramming  for  exams,  students  now  express  a different  anxiety  as 
exams  near.  Many  express  a concern  or  even  a guilt  that  they  need  not  do  much  more  to  be 
prepared  to  perform  at  a high  level.  The  students  need  reassurance  that  they  do  not  have  a false 
sense  of  security.  Continual  preparation  and  improvement  have  shifted  exams  away  from  something 
to  be  feared  to  an  opportunity  to  show  how  much  has  been  learned.  Student  contact  in  the  hours 
before  exams  has  shifted  from  panicked  pleas  to  be  saved  to  asking  for  points  of  clarification.  Often, 
exchanges  are  at  a nuanced  level  well  beyond  what  is  needed  for  the  exam. 


In  short,  online  quizzes  offer  myriad  benefits  to  increasing  student  knowledge,  understanding,  and 
retention  in  an  introductory  tax  course.  Although  the  authors'  experience  with  these  quizzes  has  been 
in  supplementing  a traditional  classroom  environment,  their  use  would  be  ideal  for  the  training  of 
technical  tax  knowledge  in  the  workforce,  particularly  if  this  training  takes  place  in  an  online 
environment.  In  online  training,  participants  often  have  difficulty  staying  on  top  of  the  material  and 
must  be  largely  self-motivated.  Implementing  pre-quizzes  provides  a way  for  participants  to  stay 
focused  and  helps  ensure  they  are  mastering  the  material  as  it  is  presented,  while  post-test  quizzes 
are  a useful  review  for  participants  to  identify  gaps  in  their  knowledge. 

Overall,  these  types  of  online  quizzes  are  useful  for  instructors  at  any  level — both  in  college  and  in 
the  broader  workforce — who  may  struggle  to  identify  misconceptions  in  their  students'  learning. 
Various  software  tools  are  available  for  creating  quizzes.  Typical  classroom  online  platforms,  such 
as  Blackboard,  include  quiz  tools. 

Editor  notes:  This  column  is  based  on  a presentation  the  authors  made  at  the  February  2015 
midyear  meeting  of  the  American  Taxation  Association  in  Washington. 
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